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No other appeals or interferences are currently known to Appellant that will directly 

affect, be directly affected by, or have a bearing on the decision to be rendered by the Board of 
Patent Appeals and Interferences in the present appeal. 


Claims L 3-15, 17-20, 22, 23, and 25-30 are pending in the application, with claims 2, 
16, 21, and 24 being cancelled. No claims have been allowed. The rejection of claims 1, 3-15, 
17-20, 22, 23, and 25-30 is the subject of this appeal. 
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IV. Status of Amendments 

No claim amendments were filed subsequent to the final rejection mailed March 22, 
2007, and all claim amendments have been entered. 

Claims 1, 3-15, 17-20, 22, 23, and 25-30 are provided in the attached Claims Appendix, 

V. Summary of Claimed Subject Matter 

Claims 1,15, and 20 are independent claims that are being appealed. 

Claim 1 is directed to a synchronization system for time-based synchronization of 
streaming media transmitted over a communications network. Referring to Appellant's 
specification, a webcast system 100 is shown in Figure 1 that is adapted for webcasting and is 
shown to include a number of streaming media sources 130, 150, 160 that provide streaming 
media inputs 1 12, 140 that may be subjected to varying transmission delays as they are 
transmitted over the communication network 138 or the Internet 144, A streaming media 
synchronization system 1 10 as called for in claim 1 is shown as part of the webcast system of 
Figure 1 for generating time-synched media signals 1 14 by processing the streaming media 
inputs 112, 140 from multiple sources. An exemplary embodiment of the synchronization 
system 1 10 is shown in detail in Figure 2. 

The synchronization system of claim 1 includes "an input interface" that, acts to receive 
first and second media stream from first and second media sources with each media stream 
including a "streaming video portion," As shown in Figure 2, two input streams 1 12 are received 
such as from sources 150, 160 (and/or source 130) by I/O interface 210 and the inclusion of 
video in the stream is discussed at least at page 1 1, lines 7-21. The system of claim 1 further 
includes first and second data buffers that store the data packets from the two media streams as 
shown at 222, 224 of Figure 2 and as explained in the paragraph beginning at line 9 of page 16. 

The synchronization system of claim 1 includes a controller "linked to the first and 
second data buffers for selectively retrieving the data packets of the first and second media 
streams to form a first and a second time-adjusted stream, wherein the controller determines a 
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variable transmission delay" for the streams from the sources to the input interface and uses this 
variable transmission delay to decide when to retrieve the buffered packets. Further, the 
controller is configured "for mixing the first and second time-adjusted streams into a composite 
media stream wherein the first and second time-adjusted streams are synchronized based on 
time," An exemplary controller is shown by the controller and media processor 230 and 240 of 
Figure 2 that act to mix time-adjusted streams 234 to output a composite stream 1 14. This 
important aspect of the system of claim 1 is explained in detail in Appellant's specification from 
page 15, line 16 to page 16, line 22, As explained, the variable transmission delay is determined 
for each of the received media streams using an external timing reference such as reference 120 
and a time stamp or information in the received stream, and the determination may be done once 
for a stream or file or two or more times for a stream or file. As can be seen from the language 
of claim 1 and this portion of the specification, the controller acts to synchronize two media 
streams that include a video portion and to also mix the two streams into a single composite 
media stream in which the video portions from differing sources are time-synchronized (e.g., as 
might be used to create side~by-side screens of live broadcasts from differing sources and/or 
geographic locations or picture-in-picture effects where the two streams are accurately time 
synchronized). 

Independent claim 1 5 is directed to an apparatus for synchronizing media streams 
transmitted over a communication network. As with the system of claim 1, the streaming media 
synchronization system 110, shown as part of webcast system 100 in Figure 1 and in more detail 
in Figure 2, is a representative embodiment of the claimed apparatus. The apparatus of claim 15 
includes an input interface, such as I/O interface 210, that acts to receive media streams 
transmitted by first and second media sources over a communications network. In contrast to 
claim 1 , the media streams of claim 1 5 are encoded to first and second compression standards 
that differ. As explained beginning at page 14, line 18, the compressed streams 212 received by 
the I/O interface 210 may be formatted "under different, incompatible compression standards 
and formatting schemes." In this regard, claim 1 5 calls for the apparatus to include a decoder 
that decodes the two received streams into intermediate streams that "are compatibly formatted." 
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The decoder may be the parsing and decoding manager 216 shown in Figure 2 that functions to 
"output compatible or codec neutral streams 220" as explained at lines 20-23 of page 14. 

Claim 15 further calls for a streaming media processor, such as processor 240 shown in 
Figure 2, for "mixing the first and the second intermediate-format media streams into a 
composite media stream encoded according to an output compression standard," For example, 
the streams 234 that are formed from codec neutral streams 220 may be mixed into a composite 
media stream 1 14 using an output compression standard, with operation of the processor 240 
explained at least at page 20, line 22 to page 21, line 21. The apparatus of claim 15 further 
includes a controller such as controller 230 of Figure 2 that determines a variable transmission 
delay for the two received media streams as discussed with reference to claim 1. Claim 15 
further calls for the determination to be done based on transmission and receipt times for a 
particular data packet in each stream as discussed at page 16, lines 1-10. 

Independent claim 20 is directed to a method with limitations similar to those of claim 1 
but presented in method form. Hence, the summary of claim 1 is applicable to claim 20. Claim 
20 further specifies that the time-adjusted streams 234 are created by correcting for the two 
determined transmission delay values by "matching the data packets of the first and second 
media streams based on transmittal times from the first and the second media sources." This 
type of matching is explained in Appellant's specification from line 9, page 16 to at least line 7, 
page 18. Claim 20 also calls for creating a synchronized media stream (e.g., mixing by processor 
240 to create composite stream 1 14) by "mixing" the two media streams and requires that the 
first and second media streams "are presented in the synchronized media stream concurrently" 
rather than sequentially or serially. This concurrent presentation is discussed at least in the 
paragraph beginning at line 15, page 21 (e.g., two or more live webcasts may be shown in 
separate windows or screens with proper time synchronization). 

VI. Granssds of Rejection to be Reviewed on Appeal 

Claims ! , 3-5, 8, 1 1 -15, 17, 18, 20, 22, 23, and 25-30 stand rejected under 35 U.S.C. 
§ 103(a) as being unpatentable over U.S. Pat. No. 5,914,757 ("Dean") in view of U.S. Pat, No. 
7,043,749 ("Davies"), 
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Claims 6, 7, and 19 stand rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Dean and Davies as applied to claims 5 and 17 further in view of U.S. Pat. No. 6,360,271 
("Schuster"). 

Claims 9 and 10 stand rejected under 35 U.S.C. § 103(a) as being unpatentable over Dean, 
Davies, and Schuster as applied to claim 6 and further in view of U.S. Pat. No. 5,313,454 
("Bustini"). 


VII. 



In the final Office Action of March 22, 2007, claims 1, 3-5, 8, 11-15, 17, 18, 20, 22, 23, 
and 25-30 were rejected under 35 U.S.C. § 103(a) as being unpatentable over U.S. Pat. No. 
5,914,757 ("Dean") in view of U.S. Pat. No. 5,914,757 ("Davies"). This rejection is traversed 
based on the following remarks, and Appellant requests that the rejection be reversed as not 
properly supported. To assist in understanding the differences between Appellant's claimed 
invention and the teaching of Davies and Dean, the remarks provide a general discussion of 
embodiments of Applicant's invention before providing a discussion of specific claim language 
and remarks that distinguish the teaching of Dean and Davies considered alone and in 
combination. 

Before turning to specific claim language, it may be useful to generally describe some of 
the significant differences between Applicant's invention and the technologies described in Dean 
and Davies. Applicant's system is effective for mixing received digital streams with video 
portions that have been streamed or transmitted over differing networks and/or different network 
paths. This is of particular importance where two or more sources of video streams are being 
combined into a single stream. Video time-based synchronization requires synchronization in 
the 1 00 nanosecond or lower range with higher definition requiring even higher levels of 
synchronization. In part, this is because video mixing may involve superimposition of two 
images or fading one image into another or cutting a hole and then matting another into the hole 
(e.g., mixing, matting, keying, and the like). For example, a live stream of a news or weather 
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anchor may be keyed into a hole created in another live stream (e.g., a stream of live action video 
or weather map or the like) during mixing operations. Such mixing may involve mapping video 
streams on a pixel-by-pixel basis or even fractions of pixels (e.g., image element in one stream to 
an image element in another stream). Such video mixing is often orders of magnitude more 
accurate than that required to match an audio signal to a video signal. 

Prior to the invention as noted in Applicant's Background, the problems associated with 
such mixing of media streams transmitted over the Internet and other networks had not been 
adequately addressed. One solution was to simply display video streams separately (e.g., avoid 
mixing) or, at most, to try to synchronize a sound stream with a video stream but again without 
mixing. Note also that the difference between mere multiplexing of video streams and the 
mixing described by Applicant was explained in detail in Applicant's September 12, 2005 
Amendment. 

There are numerous differences between the time-based mixing of received signals as 
taught by Appellant and the teaching of Dean. Generally, Dean discusses issues with taking 
video signals from two sources or input channels that may "differ in both frequency and phase" 
(col. 1, line 63) and the need for the display device to accommodate for both frequency and 
phase differences such that the source and display frequency and phase are synchronized (col. 2, 
lines 1-20) as "the display must be resynchronized to this new source signal" that can cause an 
image jump (col. 1, lines 58-60). Prior art devices are said to solve these synchronization 
problems by adj usting "the display frequency and phase to match the frequency and phase of the 
source of the primary video image" (col. 2, lines 21-27). 

Dean is directed to a device and method for improving on traditional picture-in-picture 
(PIP) display systems to avoid jumps or other issues (e.g., one screen or both black out 
momentarily) with prior art PIP devices such as that shown in Figure 1 that were used to sync 
phase and frequency of the primary image and the secondary image (the PIP image) to the 
display when changes are made between these two channels or "sources" (see, for example, col. 
6, lines 9-26). To this end, Dean teaches use of the PIP display of Figure 2 where two input 
channels or sources 161, 162 have their signals stored in buffers 140A, 140B. Then, as noted 
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beginning at col. 6, line 63, Dean teaches that "neither buffered video signal is synchronized, per 
se, to one of the channel inputs, and thus, a change of the source of one channel would not affect 
the display of the other." The sync control 220 and memory controls 1 30A, 1 308 act such that 
the synchronization signal 266 is "synchronous with the source of the image most noticeable to 
the user" (col. 7, lines 23-25) such as the larger of the two screens. The synchronization signal is 
used to match source frequencies to control video judder but constant to control video jumps and 
this paradoxical design issue for controlling frequency and phase synchronization of the two 
channel signals 161, 162 is addressed with the Phase Locked Loop embodiment of the Dean 
system described in col. 7, line 53 to col. 10, line 19. Mixer 150 is used to generate a PIP image 
for display on video display 100 based on the buffered signals 173 A, 173B, 

Significant to Appellant's invention, the Dean system provides an improvement over 
prior PIP display devices in that it provides' an improved way to match more than one input 
signal to a display devices frequency (e.g.. its read clock as compared to the source's write clock 
as discussed at col. 2, lines 1-20). However, Dean fails to discuss time-based synchronization of 
two received signals with each other. Further, there is nothing in Dean to indicate the signals on 
channels 161, 162 are not provided in standard form such as by a common television content 
provider such as a cable TV or dish TV provider. There is no discussion in Dean of problems 
associated with variable transmission delays that may occur with transmission of data packets or 
media streams over the Internet or similar networks. Instead, Dean is only concerned with 
synchronizing frequency and phase of a source with a display device and not with time 
synchronizing two concurrently displayed images . For example, Dean is not concerned with 
whether its primary image and its secondary image are displayed in a time-based manner at all 
but only that the frequencies are matched to avoid image judder or jumps. Such control does not 
require any concern over when the signals on channels 161, 162 were transmitted from an 
originating source or when they are received at the PIP display device of Figure 2 (e.g., in 
standard PIP mode, there is no concern that the two images be transmitted at a particular time). 

In contrast, Appellant provides methods and systems that combine two or more video 
streams received from differing sources (e.g.. over differing networks or paths) and with 
differing transmission times or delays into a single stream in which the streams are synchronized 
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(e.g., mapped pixel to pixel in some cases) based on transmission time from their sources. 
Frequency and phase issues discussed in Dean may still have to be addressed to create a PIP 
display without jump or judder, but this is a distinct issue relative to the time-based 
synchronization addressed by Appellant. 

Referring now to Davies, this reference is directed toward a network gateway for use in 
synchronizing audio signals delivered over a telephony network and video signals delivered over 
a digital communications network to provide "iip~synch." As discussed in prior Amendments, 
synchronizing audio and video signals is much easier than synchronizing video streams (such as 
down to the pixel level), with Davies noting in col. 9, lines 60-64 that mismatches of up to 10 to 
40 milliseconds are acceptable. Davies is attempting to solve a time-based synchronization 
problem, but it does not teach synchronizing two video streams and Davies attacks the problem 
very differently than Appellant even in the audio to video realm. Specifically, Davies teaches in 
col. 10 that it is assumed that "network propagation delays are predictable" (col. 10, lines 25-30) 
in the design of the gateway and that "network propagation delays may be assumed to be 
constant during a call as users can cope with minor blips in synchronization" (col. 10, lines 46- 
48). In contrast, Appellant's invention is directed to solving problems associated with "non- 
deterministic delays" and "variable transmission rates" that may differ and change over time for 
each media stream or even on a packet-by -packet basis (e.g., even from a single source the path 
of data packets may vary within the stream or from stream to stream). 

With these assumptions in mind, Davies teaches in its Summary of the Invention, in col. 
12 and elsewhere, a calibration technique for particular video equipment, compression and 
encoding algorithms, and transmission of test audio and video signals. For the particular system 
configuration, a mismatch is determined for the synchronization of an audio signal and a video 
signal, and a "variable delay" in the gateway is adjusted until the two signals are synchronized. 
Then, with reference to Figure h the calibrated gateway 101 is used to apply this variable delay 
on a signal routed through the gateway, such as the audio signal or the video signal, when one of 
the combined multimedia terminals are used. There is no ongoing adjustment for variable 
transmission delays as the "variable delay" is stored "in the gateway for future use" (col. 2, line 
45) and "only new combinations of audio and video devices on different networks will need 
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calibration" (col. 12, lines 4-8). Hence, Davies teaches a one-time calibration technique for 
accounting for differing delivery rates for an audio signal and a video signal, such as may be 
useful for video conferencing. 

However, when compared generally with Appellant's invention, there is no teaching in 
Davies of determining transmission delay for both of two received streams as Davies only 
teaches tuning or adjusting a delay applied to a signal passing through a gateway device until 
synchronization is achieved one time and then storing that delay value (e.g., that actual delay of 
either of the signals does not have to be determined). As a result, there is no teaching of looking 
at information in the video streams themselves to determine such a delay value. Further, Davies 
provides no teaching that the two signals are mixed or combined in any manner as the stored 
variable delay is simply applied to one of the signals but both signals are delivered to their 
separate terminals (e.g., audio terminal 104 and video terminal 105 of a combined multimedia 
terminal 108). Hence, there is no teaching of placing the two signals into a neutral form prior to 
storage in a buffer to facilitate later combination into a sequential or concurrent composite media 
stream. 

Turning now to the claim language, claim 1 calls for an input interface that receives first 
and second media streams that each include a streaming video portion. According to claim 1, the 
two media streams "comprise a plurality of digital packets being transmitted over the 
communications network from a first and a second media source." A controller determines a 
"variable transmission delay for the first and second media streams from the first and second 
media sources to the input interface and performs the selective retrieving based on the 
determined variable transmission delays." Additionally, the controller mixes the retrieved time- 
adjusted streams "into a composite media stream wherein the first and second time-adjusted 
streams are synchronized based on time." The combined teaching of Dean, and Davies fails to 
teach or suggest each of these limitations of claim 1. 

The final Office Action cites Dean for teaching the input interface of claim 1 at col. 4, 
lines 6-22 and Figure 2 elements 161, 162. Appellant disagrees with this assertion because Dean 
teaches a conventional PIP display that is combining 2 signals based on frequency and phase, but 
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there is no suggestion in Dean that the input signals "comprise a plurality of digital data packets 
being transmitted over the communications network from a first and a second media source," 
This is significant to claim 1 because it is the use of a communications network by two media 
sources that are providing the first and second signals that creates the issue of two differing 
transmission delays. The portion of col, 4 cited by the Examiner states that typically the "two 
channel inputs are IN- A 161 and IN-B 162" which typically would come from two television 
channel tuners (i.e.. from a single "source" such as a cable line or dish receiver and a content 
provider). However, Dean at this citation then states that "such signals may have come from two 
independent sources of video images, as well, such as a video camera and a computer device," 
This statement appears to have more to do with the fact that the source frequency and phase may 
differ for the two "sources," Therefore, Dean does not teach: (a) that the signals on channel 
inputs IN- A 161 and IN-B 162 are digital data packets that have been transmitted, over a 
communications network or (b) that the packets were transmitted over such a network from two 
differing media sources. As a result, the input interface element of claim 1 is missing from the 
teaching of Dean. 

The Advisory Action fails to address this and other distinguishing remarks made by- 
Appellant in the Amendment filed July 23, 2007. Instead, the Advisory Action asserts that 
Appellant is only arguing against Dean and Davies individually and not with regard to their 
combined teaching. Appellant disagrees with this statement because the July 23, 2007 
Amendment clearly states the failings of Dean (which is the primary 103 reference) and then 
Appellant explains how Davies fails to overcome each of these particular failings or defi ciencies 
of Dean. In other words, the combination of the two references fails to produce the claimed 
invention because one or more elements are missing in both references. For example, Dean fails 
to teach the interface limitation of claim 1 , and Davies fails to overcome this deficiency (even 
though it was not cited for overcoming this deficiency) because it teaches a gateway for 
synchronizing delivery of an audio signal with a video signal. Davies makes no mention of an 
interface for receiving first and second media streams with a streaming video portion from first 
and second media sources over a communications network. Because neither of the two cited 
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references teach the input interface of claim 1, these references fail to support an obviousness 
rejection of claim 1. 

Further, Dean and Davies fail to teach the controller of claim 1 with consideration of the 
final wherein clause which further defines the controller. The final Office Action cites Dean as 
teaching the controller of claim 1 that functions "for mixing the first and second time-adjusted 
streams into a composite media stream wherein the first and second time-adjusted streams are 
synchronized based on time." But, in the first full paragraph of page 4 the final Office Action 
states that Dean does not. disclose "the time-adjusted streams are synchronized based on time." 
Appellant asserts that the second conclusion of the Examiner is correct based on a proper 
construction of Dean, 

The final Office Action cites Dean at col, 4, lines 51-65 and mixer 150 of Figure 2 for 
supporting the first conclusion, which Appellant argues is a conflicting and an improper 
conclusion. The only line that seems relevant at this citation states the "mixer 150 presents the 
appropriate portion of the image to be displayed at the time the display requires it" by retrieving 
images from the buffers. However, Dean is directed toward frequency and phase 
synchronization of input signals with a display device so the "time" discussed here appears to be 
referring to retrieving a buffered image using a synchronization signal when a PIP display is 
demanded and placing the images in the PIP display in the primary or secondary portions as 
required. As discussed above, there is no discussion of network propagation delays or of 
synching the two images from the two input channels based on when they were transmitted from 
their sources. With this in mind, Appellant asserts that the controller limitation in light of the 
final wherein clause of claim 1 is not shown by Dean. 

The Advisory Action argues that "Dean's system also synchronizes the multiple videos 
sources by adjusting the time (see Dean col. 5, lines 1-7)" but fails to teach "the available 
transmission delay." Appellant strongly disagree with this interpretation of Dean (as discussed 
in the preceding paragraph). Appellant's claim 1 calls for determining a transmission delay for 
both the first and second media streams so that synchronizing based on transmittal time may be 
performed. If this is never calculated by a reference such as Dean, the reference fails to teach 
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time-adjusted streams that can be mixed as called for in claim 1 . The citation to col. 5 is 
discussing the adjusting of time a secondary video signal is presented to a mixer to synchronize 
phase when the frequency is already the same. However, there is no discussion of calculating a 
transmission delay or of synchronizing the primary and secondary video signals based on their 
transmittal times. Instead, Dean only discusses synchronization of input signals based on their 
frequency and phase to avoid judder when the PIP channels are switched. As a result, Dean fails 
to teach the controller of claim 1 because this device functions to mix first, and second "time- 
adjusted streams" (with time adjustment being based on a determined variable transmission delay 
for each stream in claim 1 and not a retrieval from a buffer to synchronize phase/frequency 
between a source and a display) into composite media stream. In other words, the limitation 
"time-adjusted stream" of claim 1 must be construed in the context of the claim language to 
includes the use of determined variable transmission delays to form these streams whereas the 
time discussed in col. 5 of Dean pertains to time of withdrawal from a buffer to adjust a phase of 
a signal delivered to a display. As a result. Dean fails to teach the controller of claim 1 and is 
deficient for more than missing the "available transmission delay" as asserted in the Advisory 
Action. 

The Office Action on page 4 states that Dean fails to disclose "selectively retrieving the 
data packets of the first and second media streams to form a first and a second time-adjusted 
stream, the controller determines a variable transmission delay for the first and the second media 
streams from the first and second media sources to the input interface and performs the selective 
retrieving based on the determined variable transmission delays, and the time-adjusted streams 
are synchronized based on time." Hence, Davies is required to show each of these features of the 
controller and also to overcome the input interface deficiency of Dean discussed above. 

First, Applicant asserts that Davies fails to show "the controller determines a variable 
transmission delay for the first and the second media streams from the first and second media 
sources to the input interface," The Office Action cites Davies at col 4, lines 1 1-30 and lines 
45-63, and col. 15, lines 22-46. Initially, in Dean, "variable delay" is described in col. 2, lines 
56-67 as the delay in the sensory audible and visible output from the separate output devices 
(such as a speaker or video display unit). Any detected mismatch in the synchronization of these 
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outputs is accounted for by adjusting the variable delay applied by the gateway on one of the 
signals (e.g., typically the audio signal) to achieve a calibrated synchronization (and the delay is 
then applied whenever the calibrated device is used). However, in Dean's Summary, there is no 
discussion of calculating the transmission delay for the data packets in the first stream due to 
their travel over the network and the transmission delay for the data packets in the second stream 
due to their travel over the network, e.g., claim 1 requires that "delays" be used to perform the 
selective retrieving and not a single value. 

Referring to the specific citations, at col. 4, lines 1 1-30, Davies mentions a 
synchronization delay that may involve a transit delay or propagation delay, but such a delay is 
only between the gateway device and the audio or video device and not over a network from two 
differing media sources . Also, Davies beginning at line 19 of col. 4 quickly returns to the 
discussion of a sensory output delay that is "defined as the time difference between the audio and 
video the user perceives. at the terminal ." It is this perceived difference that is accounted for and 
not a determined transit delay or as called for in claim 1 a "variable transmission delay." Davies 
teaches measuring the "actual delay between the audible and visible output, and then by storing a 
value corresponding to this variable delay in the gateway" so that one of the signals can be 
delayed in the gateway device to achieve a better synchronization of delivered signals in a 
calibrated manner. Hence, Davies fails to teach a controller that determines a variable 
transmission delay for two media streams between a source and an interface as called for in 
claim 1. 

Second, Appellant asserts that Davies fails to show a controller selectively retrieves "the 
data packets of the first and second media streams to form a first and a second time-adjusted 
stream" wherein the selective retrieving is "based on the determined variable transmission 
delays." As discussed above, Davies fails to teach determining the variable transmission delays 
as called for in claim 1, and, as a result, Davies cannot teach selective retrieval of data packets 
from buffers based on these values. However, even if the variable delay discussed in Davies 
taught these two values, Davies would still fail to teach the selective retrieval of claim 1. This is 
because Davies teaches in its Summary and elsewhere that once a calibrated "variable delay" is 
stored in the gateway device that it is applied to either the audio or the video signal to achieve 
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synchronization. This same delay is imposed by network gateway whenever the system is used, 
e.g., until new components are added. There is no teaching of retrieving two streams of data 
based on variable delays but instead Davies teaches slowing one of the two streams down a 
particular amount to achieve synchronization at the output devices (e.g., the audio and video 
terminals of the combined media devices). 

The Office Action cites col. 4, lines 1 1-30 and 45-63 and col. 15, lines 22-46 for 
providing the selective retrieval, but lines 1 1-30 discuss the "variable sensory output delay" and 
how to control it by "either the audio or the video signal is delayed within the gateway by the 
appropriate amount" (where is selective retrieval from a buffer shown?). At col. 4, lines 46-63 
and at col. 15, lines 22-46, Davies discusses how to account for processing delays (e.g., skew) 
within the gateway device itself in loop back type systems with some of these delays being 
"known from standard techniques for that type of terminal" (rather than varying each time 
packets are sent on a network). Hence, the cited portions of Davies fail to teach a controller 
performing the selective retrieval called for in claim 1 to form two time-adjusted streams from 
buffers. 

In summary, Appellant disagrees with the Examiner that Davies teaches the controller 
features or functionality for which it is cited in the final Office Action. Further, Davies fails to 
overcome the deficiency discussed above for Dean in that there is no showing of an input 
interface that receives two media streams transmitted from two media sources and comprising 
streaming video portions. Davies, instead, teaches receiving an audio signal from a telephony 
network and receiving a video signal such as from a LAN, a WAN, or via the Internet, Hence, 
Dean and Davies when combined fail to teach or suggest the system of claim 1 because their 
combined teaching fails to show either the input interface or the controller that functions as 
required to generate a composite media stream that is formed from two time-adjusted streams 
such that these two streams are synchronized based on time. Applicant requests that the rejection 
of claim 1 be withdrawn as not properly supported. 

The Advisory Action implies that when considered in isolation neither reference teaches 
all of the features of the claimed invention, but when Dean and Davies are considered together 
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the invention is shown. Appellant disagrees because combining the teaching of these two 
references would not result in the claimed system (i.e., there is no or little likelihood of 
successfully producing the claimed invention by combining the teaching of the 2 references). 
Dean teaches synching the frequency of two video signals for two input channels selected for 
PIP display with the frequency of the display' s output but not providing a synchronization of the 
two video signals based on their transmission times or a variable delay. Davies teaches 
calibrating with a gateway device an audio output and a video output (again, this appears to be a 
one time calibration that is not applied to two streams of data packets based on non-deterministic 
or continually varying network propagation delays) by delaying one of the two signals in an 
ongoing, constant fashion for that group of components. If Dean were modified based on 
Davies, the result likely would be a PIP display with images from two input channels that are 
provided at frequencies that are synched with the frequency of the display and the display device 
would be combined with a separate audio output device without mixing. But, there would, still 
be no teaching that it was important to try to achieve time-based synchronization of the two 
images of the PIP display (e.g., why does a viewer care if a football game on the ESPN channel 
is synchronized to account for transmission delay with a sitcom on the NBC channel whereas 
you do want to avoid jumps, judder, or other issues when flipping between the two images as 
primary/secondary?) or how such synchronization would be achieved let alone the techniques 
called for in claim 1 . The only motivation to provide time-based synchronization between the 
two video signals of Dean on the input channels presented for PIP display is Appellant's own 
specification as Davies only discusses lip synching an audio signal with a video signal and not 
two video signals while Dean provides no discussion of time-based synchronization. For these 
additional reasons, Appellant believes these two references fail to make the system of claim 1 
obvious to one skilled in the arts. 

In the examination of a patent application, the Examiner bears the initial burden of 
showing a prima facie case of unpatentability, and Appellant asserts that the Examiner has not 
even carried this initial burden with regard to claim 1 because Dean fails to teach the "input 
interface" as called for in claim 1 and the combination of Davies and Dean fail to teach the 
"controller" of claim 1 . Since the Examiner has not cited two or more references showing or 
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suggesting each and every element of the claim, a proper prima facie case of obviousness has not 
been presented, However, even if it is found that the Examiner has met this initial burden, the 
Appellant believes the arguments presented in the preceding paragraphs provide rebuttal 
evidence with more than adequate weight to show the Examiner has erred and to dissipate or 
overcome the Examiner's prima facie case of unpatentability. Appellant, therefore, requests the 
Board review the entire record and make a finding that the sy stem of claim 1 is patentable. 

Claims 3-5, 8, and 11-14 depend from claim 1 and are believed allowable over Dean and 
Davies at least for the reasons provided for allowing claim 1. Further, claims 3 and 4 are 
addressed to cases where the two received media streams have differing compression formats 
and handling the streams to allow mixing by decoding into compatible forms. There is no 
discussion of decoding the two streams into formats that are compatible prior to storage in 
buffers in Dean or Davies. The final Office Action cites Dean at col. 10, lines 31-46 for teaching 
the limitations of claims 3, which states that the two media streams may be compressed on first 
and second compression formats that differ. However, at the cited portions, Dean states the 
shown configuration "allows for the video images to be processed and stored in an optimized 
form, for example, as compressed video, for synchronization purposes, then present in detailed 
form, for example, as uncompressed video, for display." This does not teach that the signals on 
the two inputs 161 and 162 are received with differing compression formats but, instead, seems 
to teach buffering the two signals as compressed video and then processing for display as 
uncompressed video. 

Davies is cited at co. 4, lines 1 1-22 for teaching the limitations of claim 4, which calls for 
a decoding device that creates first and second decoded streams from the two incoming streams 
that are compatibly formatted prior to storage in the buffers. As discussed previously, there is no 
mixing/combining of the audio and video signals in Davies; so, there is no reason to format them 
in compatible formats. At col. 4, lines 1 1-22, Davies is discussing sensory output delay and 
provides no teaching whatsoever regarding decoding the two signals into compatible formats 
prior to storage in buffers. For these additional reasons, claims 3 and 4 are believed allowable 
over Dean and Davies. 
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Claim 5 calls for the controller to "form the composite media stream by combining the 
first and second time-adjusted streams such that the second time-adjusted stream has a first data 
packet positioned at a start time adjacent a last data packet of the first time-adjusted stream 
positioned at an end time." This is to avoid dead air between two video streams. Dean is cited 
for teaching this limitation at col. 4, lines 51-65. However, Dean is concerned with PIP display 
and provides no teaching of forming composite, sequential video streams. Instead, Dean teaches 
displaying primary or secondary images in PIP arrangement. At col. 4 S lines 51-65 the term 
"sequential" is used but this is only in reference to how video displays are generated line-by-line 
from top to bottom and this would be true for a mixed PIP display. But, the PIP image would not 
be created by placing the start time of one image stream next to the l ast data packet of the other 
image stream or this would result in one of the images being null or black (e.g., the secondary or 
primary image would disappear on the display screen). Hence, Dean fails to support a rejection 
of claim 5 for this additional reason. 

Claim 8 calls for a data parsing device that retrieves time data from the two media 
streams that is used by the controller to determine variable transmission delays for the two 
streams. Davies is cited at col. 4, lines 1 1-30 and lines 45-64 and col. 15, lines 22-46 for 
teaching this limitation. Initially, it should be remembered that, Davies fails to teach receiving 
and processing two video streams for later mixing/combination, and, hence, this reference cannot 
teach this limitation. Further, at col. 4, Davies discusses measuring time differences and "actual 
delay." but there is no teaching of retrieving time data such as a time stamp or transmission time 
from received data packets for use in determining these time differences or actual delay. In col. 
15, Davies discusses transmission of a marker signal to assist in determining skew caused by the 
gateway device but does not discuss retrieving "time data" from media streams. Since there is 
no teaching or discussion of the limitations of claim 8 in Davies, the rejection of claim 8 should 
be withdrawn for this additional reason. 

Independent claim 15 is directed to an apparatus for synchronizing media streams 
transmitted over a communication network. As with the system of claim 1, the streaming media 
synchronization system 110 shown as part of webcast system 100 in Figure 1 and in more detail 
in Figure 2 is a representative embodiment of the claimed apparatus. The apparatus of claim 1 
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includes an input interface, such as I/O interface 210, that acts to receive media streams 
transmitted by first and second media sources over a communications network. In contrast to 
claim 1, the media streams are encoded to first and second compression standards that differ. As 
explained beginning at page 14, line 18, the compressed streams 212 received by the I/O 
interface 210 may be formatted "under different, incompatible compression standards and 
formatting schemes." In this regard, claim 15 calls for the apparatus to include a decoder that 
decodes the two received streams into intermediate streams that "are compatibly formatted," 
The decoder may be the parsing and decoding manager 216 shown in Figure 2 that functions to 
"output compatible or codec neutral streams 220" as explained at lines 20-23 of page 14. 

Claim 1 5 further calls for a streaming media processor, such as processor 240 shown in 
Figure 2, for "mixing the first and the second intermediate-format media streams into a 
composite media stream encoded according to an output compression standard." For example, 
the streams 234 that are formed from codec neutral streams 220 may be mixed into a composite 
media stream 114 using an output compression standard, with operation of the processor 240 
explained at least at page 20, line 22 to page 21, line 21. The apparatus of claim 15 further 
includes a controller such as controller 230 of Figure 2 that determines a variable transmission 
delay for the two received media streams as discussed with reference to claim 1 . Claim 15 
further calls for the determination to be done based on transmission and receipt times for a 
particular data packet in each stream as discussed at page 16, lines 1-10. 

Independent claim 15 includes limitations similar to those provided in claim 1. Hence, 
the reasons provided above for allowing claim 1 over Dean and Davies are believed applicable to 
claim 15 and are incorporated herein. Additionally, claim 15 includes limitations similar- to 
dependent claims 3 and 4, and the reasons for allowing claims 3 and 4 over Dean and Davies are 
applicable to claim 15 and incorporated herein. 

Claims 17 and 18 depend from claim 15 and are believed allowable over Dean and 
Davies at least for the reasons provided for claim 15. 

Independent claim 20 was rejected for substantially the same reasons as provided for 

claim 1 . Independent claim 20 is directed to a method with limitations similar to those of claim 
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1 but presented in method form. Hence, the description of claim 1 provided above is applicable 
to claim 20. Claim 20 further specifies that the time-adjusted streams 234 are created by 
correcting for the two determined transmission delay values by "matching the data packets of the 
first and second media streams based on transmittal times from the first and the second media 
sources " (emphasis added). This type of matching is explained in Appellant's specification from 
line 9, page 16 to at least line 7, page 1 8. Claim 20 also calls for creating a synchronized media 
stream (e.g., mixing by processor 240 to create composite stream 1 14) by "mixing" the two 
media streams and requires that the first and second media streams "are presented in the 
synchronized media stream concurrently" rather than sequentially or serially. This concurrent 
presentation is discussed at least in the paragraph beginning at line 15, page 21 (e.g., two or more 
live webcasts may be shown in separate windows or screens with proper time synchronization). 

Because claim 20 includes limitations similar to claim 1, the reasons for allowing claim 1 
over Dean and Davies are believed applicable to claim 20. Additionally, claim 20 calls for the 
two media streams to include data packets from one or more video files and adjusting the two 
streams including "matching the data packets of the first and second media streams based on 
transmittal times from the first and second media sources," Dean and Davies fail to show the use 
of transmittal times for either their frequency/phase synchronization of PIP images or for the 
delay of an audio signal in a gateway to match a video signal. The Office Action states that 
Davies teaches the retrieval of timing data from two media streams, but the citations to Davies 
fail to support such a statement as was discussed above with reference to dependent claim 8. The 
Advisor}' Action failed to address this portion of Appellant's argument. Further, claim 20 
specifically calls for using "transmittal times from the first and second media sources." Davies 
fails to teach determining any of its delays based on transmittal time with most of its discussion 
going toward measuring actual differences between the timing of outputs on audio and video 
terminals or devices. Hence, Dean and Davies fail to support a rejection of claim 20 for this 
additional reason. 

Claims 22, 23, and 25-30 depend from claim 20 and are believed allowable over Dean 
and Davies for at least the reasons provided for allowing claim 20. Further, claims 25 and 26 
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have limitations similar to claims 3 and 4 and are believed allowable for the n 
these claims. 


s provided for 


r 35 U.S.C. §103 is I 


Claims 6, 7, and 19 stand rejected under 35 U.S.C. 103(a) as being unpatentable over 
Dean and Davies as applied to claims 5 and 17 further in view of U.S. Pat. No. 6,360,271 
("Schuster"). Claims 6 and 7 depend from claim 1 and claim 19 depends from claim 15, and 
these claims are believed allowable over Dean and Davies at least for the reasons provided for 
claims 1 and 15. Schuster is not cited for overcoming (and fails to overcome) the deficiencies 
pointed out for Dean and Davies with regard to claims 1 and 15, and, as a result, claims 6, 7, and 
19 are believed allowable over the combined teaching of Dean, Davies, and Schuster. 


r35 U.S.C. §103 la I 


Claims 9 and 10 stand rejected under 35 U.S.C. 103(a) as being unpatentable over Dean, 
Davies, and Schuster as applied to claim 6 further in view of U.S. Pat. No. 5,313,454 
("Schuster"). Claims 9 and 10 depend from claim 6 and are believed allowable for over Dean, 
Davies, and Schuster at least for the reasons provided for allowing claim 6. Further, Bustini is 
not cited as overcoming (and fails to overcome) the deficiencies of Dean and Davies with regard 
to base claim 1. Hence, claims 9 and 10 are believed allowable over the combined teaching of 
these four cited references. 
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In view of the above remarks, the pending claims are believed allowable and the case in 
condition for allowance. Appellant respectfully requests that the rejections of all pending claims 
be reversed as Appellant has met or exceeded the burden of overcoming the prima facie case of 
unpatentability made out by the Examiner by providing rebuttal evidence of adequate weight and 
because the record indicates the pending claims are patentable. 

Respectfully submitted, 



Marsh Fischmann & Breyfogle LLP 

1881 9 th Street, Suite 335 
Boulder, Colorado 80302 
Phone: (720) 562-5507 
Fax: (720) 562-5519 
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Vffl, CLAIMS APPENDIX 

1 . A synchronization system for time-based synchronization of streaming 
media transmitted over a communications network, comprising: 

an input interface adapted for linking to the communications network to receive 
a first and a second media stream, wherein the first and second media streams 
comprise a plurality of digital data packets being transmitted over the communications 
network from a first and a second media source, respectively, and wherein the first and 
the second media streams each include a streaming video portion; 

a first data buffer for storing the data packets of the first media stream; 

a second data buffer for storing the data packets of the second media stream; 

and 

a controller communicatively linked to the first and the second data buffers for 
selectively retrieving the data packets of the first and second media streams to form a 
first and a second time-adjusted stream, wherein the controller determines a variable 
transmission delay for the first and the second media streams from the first and second 
media sources to the input interface and performs the selective retrieving based on the 
determined variable transmission delays; 

wherein the controller is further configured for mixing the first and second 
time-adjusted streams into a composite media stream wherein the first and second 
time-adjusted streams are synchronized based on time. 

3, The system of claim 1, wherein the streaming video portion of the first 
media stream is compressed based on a first compression format and the second media 
stream is compressed based on a second compression format, the second compression 
format differing from the first compression format. 
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4. The system of claim 3, further including a decoding device between the 
input interface and the first and second data buffers for processing compressed first 
and second media streams into a first decoded stream and a second decoded stream, 
respectively, for storage in the first data buffer and the second data buffer, wherein the 
first decoded stream and the second decoded stream have compatible formatting. 

5. The system of claim 1, wherein the controller forms the composite media 
stream by combining the first and the second time-adjusted streams such that the 
second time-adjusted stream has a first data packet positioned at a start time adjacent a 
last data packet of the first, time-adjusted stream positioned at an end time. 

6. The system of claim 5, wherein the controller is communicatively-linked to 
an external timing reference for receiving a reference time value, and wherein the 
controller is adapted for using the reference time value to determine the start time and 
the end time. 

7. The system of claim 5, wherein the controller determines a length of the first 
media stream, compares the length with the end time and the variable network delay, 
computes an edit length for the first media stream, and compresses or lengthens the 
first media stream to form the first time-adjusted stream, whereby the last data packet 
coincides with the end time. 

8. The system of claim 1, further including a data parsing device in 
communication with the input interface configured for retrieving time data from the 
first and the second media streams and for transmitting the time data to the controller, 
wherein the controller uses the time data to determine variable transmission delays. 

9. The system of claim 7, wherein the controller is adapted to create media 
server control signals based on the determined variable transmission delays and to 
transmit the signals over the communications network to the first and the second media 
sources to control transmission variables of the first and second media streams. 
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10. The system of claim 9, wherein the transmission variables are selected 
from a group consisting of transmission timing, transmission rate, and transmission 
length. 

! ! . The system of claim 1, wherein the composite media stream comprises a 
streaming video portion having picture-in-picture or side by side portions formed with 
the data packets of the first and the second time-adjusted streams. 

1 2. The system of claim 1 , wherein the controller combines the first media 
stream and second media stream in the composite media stream such that a data packet 
transmitted in the first media stream from the first media source at a transmission time 
is matched with a corresponding data packet in the second media stream transmitted 
from the second media source at the transmission time. 

13. The system of claim 12, wherein the combining is performed by the 
controller by selecting a transmission rate for the first and the second media streams to 
correct for the determined variable transmission delays. 

14. The system of claim 1, further including an output interface for 
transmitting the composite media stream from the controller over the communications 
network and including an end-user node linked to the communications network for 
receiving the composite media stream, wherein the end-user node comprises a 
synchronizer for determining a variable transmission delay between the controller and 
the end-user node and for performing time-based correction of the composite media 
stream to adjust for the variable transmission delay. 
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15. An apparatus for synchronizing media streams transmitted over a 
communication network, comprising: 

an input interface linked to the communications network and configured for 
receiving a first and a second media stream transmitted by a first and a second media 
source, respectively, wherein the first media stream comprises a plurality of data 
packets of a video stream encoded to a first compression standard and the second 
media stream comprises a plurality of data packets of a video stream encoded to a 
second compression standard differing from the first compression standard; 

a decoder for decoding the first and the second media streams into a first and a 
second intermediate media stream, respectively, wherein the first and second 
intermediate streams are compatibly formatted; 

a streaming media processor for mixing the first and the second intermediate- 
format media streams into a composite media stream encoded according to an output 
compression standard; and 

a controller in communication with the input interface and the streaming media 
processor adapted for determining a variable transmission delay for the first and the 
second media streams based on a transmission time for a data packet of the first media 
stream and a time of receipt at the input interface of the data packet and on a 
transmission time for a data packet of the second media stream and a time of receipt at 
the input interface of the data packet. 

1 7. The apparatus of claim 15, wherein the controller is further configured for 
adjusting the first intermediate-format media stream based on the variable transmission 
delay of the first media stream and for adjusting the second intermediate-format media 
stream based on the variable transmission delay of the second media stream to create a 
first and a second time-adjusted stream. 
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18. The apparatus of claim 17, wherein the processor combines the first and 
second time-adjusted stream to form the composite media stream with the first media 
stream data packet and the second media stream data packet being positioned for 
concurrent delivery. 

19. The apparatus of claim 17, wherein the time of receipt is determined based 
on a time reference signal received from an external timing reference. 

20. A method for time-based synchronization of two or more media streams 
transmitted over a data communications network, comprising: 

receiving a first media stream comprising a plurality of data packets from one 
or more video files transmitted over the communications network by a first media 
source; 

receiving a second media stream comprising a plurality of data packets from 
one or more video files transmitted over the communications network by a second 
media source; 

retrieving timing data from the first and second media stream; 

comparing the timing data with a reference time to determine a first and a 
second transmission delay value; 

adjusting the first and the second media streams to correct for the first and the 
second transmission delay values, wherein the adjusting includes matching the data 
packets of the first and the second media streams based on transmittal times from the 
first and the second media sources; and 

creating a synchronized media stream by mixing the first and the second media 
streams, wherein the first and the second media streams are presented in the 
synchronized media stream concurrently. 

22. The method of claim 20, wherein the adjusting includes creating a first and 
a second control signal in response to the first and the second transmission delay 
values, respectively, and includes transmitting the first and the second control signals 
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to the first and the second media source to control transmittal of the first and the 
second media streams. 

23. The method of claim 20, storing the data packets of the first media stream 
in a first data buffer and the data packets of the second media stream in a second data 
buffer and wherein the adjusting includes selectively retrieving the data packets of the 
first media stream from the first data buffer to correct for the first transmission delay 
value and selectively retrieving the data packets of the second media stream from the 
second butter to correct for the second transmission delay value. 

25. The method of claim 20, wherein the first media stream is encoded to a 
first compression standard and the second media stream is encoded to a second 
compression standard, and further including forming a first intermediate data stream 
by decoding the first media stream and forming a second intermediate data stream by 
decoding the second media stream, wherein the first and second intermediate data 
streams are compatibly formatted. 

26. The method of claim 25, wherein the first compression standard differs 
from the second compression standard such that the first and second media stream are 
incompatible prior to the decoding to be combined into a single media stream. 

27. The method of claim 20, wherein the synchronized media stream 
concurrently provides two screens corresponding to each of the first and second media 
streams. 

28. The method of claim 27, wherein the two screens are arranged as split 
screens. 

29. The method of claim 27, wherein the two screens are arranged as picture- 
in-picture with a first of the two screens provided within a second of the two screens. 

30. The method of claim 20, wherein the first media stream comprises a first 
live webcast and the second media stream comprises a second live webcast. 
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K. EVIDENCE APPENDIX 

No copies of evidence are required with this Appeal Brief. Appellant has not 
relied upon any evidence submitted under 37 C.F.R. §§ 1.130, 1.131, or 1.132. 
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X. RELATED PROCEEDINGS APPENDIX 

There are no copies of decisions rendered by a court or the Board to provide with 
this Appeal as there are no related proceedings. 
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